<html><head><style></style></head><body><center><h1>Online Result Summary</h1></center><h3>Model: densenet_onnx</h3><div style="font-size:14"><p>GPU(s): 8 x Tesla V100-SXM2-16GB</p></div><div style="font-size:14"><p>Total Available GPU Memory: 126.4 GB</p></div><div style="font-size:14"><p>Constraint targets: None</p></div><div style="font-size:14"><p>In 52 measurements across 6 configurations, <strong>densenet_onnx_config_3</strong> provides the best throughput: <strong>1683 infer/sec</strong>.<br><br>This is a <strong>91% gain</strong> over the default configuration (882 infer/sec), under the given constraints on GPU(s) 8 x Tesla V100-SXM2-16GB.<UL><LI> <strong>densenet_onnx_config_3</strong>: 32 GPU instances with a max batch size of 0 on platform onnxruntime </LI> </UL></p></div><div style="font-size:14"><p>Curves corresponding to the 3 best model configuration(s) out of a total of 6 are shown in the plots.</p></div><center><div><div class="image" style="float:center;width:66%"><img src="" style="width:100%"><center><div style="font-weight:bold;font-size:12;padding-bottom:20px">Throughput vs. Latency curves for 3 best configurations.</div></center></div></div></center><center><div><div class="image" style="float:center;width:66%"><img src="" style="width:100%"><center><div style="font-weight:bold;font-size:12;padding-bottom:20px">GPU Memory vs. Latency curves for 3 best configurations.</div></center></div></div></center><div style="font-size:14"><p><div style = "display:block; clear:both; page-break-after:always;"></div>The following table summarizes each configuration at the measurement that optimizes the desired metrics under the given constraints.</p></div><center><table style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt"><tr><th style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">Model Config Name</th><th style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">Max Batch Size</th><th style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">Dynamic Batching</th><th style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">Instance Count</th><th style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">p99 Latency (ms)</th><th style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">Throughput (infer/sec)</th><th style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">Max GPU Memory Usage (MB)</th><th style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">Average GPU Utilization (%)</th></tr><tr><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">densenet_onnx_config_3</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">0</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">Disabled</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">32:GPU</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">42.722</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">1682.82</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">2299</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">58.2</td></tr><tr><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">densenet_onnx_config_2</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">0</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">Disabled</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">24:GPU</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">82.172</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">1663.2</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">2128</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">81.3</td></tr><tr><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">densenet_onnx_config_4</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">0</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">Disabled</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">40:GPU</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">48.583</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">1580.74</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">2459</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">59.1</td></tr><tr><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">densenet_onnx_config_default</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">0</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">Disabled</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">8:GPU</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">18.742</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">881.772</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">1782</td><td style="border: 1px solid black;border-collapse: collapse;text-align: center;width: 80%;padding: 5px 10px;font-size: 11pt">36.1</td></tr></table></center></body></html>